Supporting backups for a high-density file system using traditional mechanism of walking through the files and collecting information can often be very slow. Snapshot backup programs, such as the Networker SnapImage Module solve this problem by taking snapshot of the file system to perform live backups at block level. Block level systems support volume managers that back up volumes, which may comprise multiple disks. These volume managers may support various data distribution formats, such as concatenated, striped, RAID, mirrors, and so on. Mirror formats provide a level of data protection and failover or switchover capability over the simple striped or concat data distribution format. Reading file system data at the block level thus requires a thorough understanding of disk data organization, and with ever changing layouts of disk data organization, this creates new challenges for snapshot backup SnapImage programs. Changes in layout formats that are introduced with newer software versions or newer operating systems increase these challenges even further.
Existing solutions for block-level incremental backup systems are built around discovering the storage stack from the volume manager pseudo device till the lowest level disk. The backup and recover operations are performed using the lowest disk and input/output (I/O) calls are intercepted at the same level. Although this solution may work in most of the situations, a high degree of software maintenance is required to ensure proper operation.
A typical storage stack production environment consists of a file system (FS) that is supported by a volume manager (VM). Present volume managers support the common disk layouts of concatenated, striped, mirror, and RAID. For high availability or load balancing each device may be seen through more than one path by the volume manager such as in multi-pathing solutions provided by Veritas dynamic multipath (VxDMP) and EMC (PowerPath). In order to support block level backups, backup vendors need to understand the volume manager layout structures, i.e., concatenated, striped, mirror, etc. These layout formats change across operating systems and volume managers. To ensure proper operation, backup vendors must also understand the failovers handled by volume manager and the ones happening at the device path level, and the load balancing that happens at the volume manager and at the disk path level, as well as have an understanding of advance volume manager features like hot swap-able or relocation of devices.
In general, this level of understanding must be built around the whole storage stack environment. Because there is no standardized implementation, however, each vendor tends to follow its own propriety specification. The approach changes with different operating systems and even between different versions of the same operating system. For example, Solaris may not follow what HP-UX guides or implements with regard to the storage stack. Furthermore, the lowest level disk technology changes across operating systems and may change with scalability. The discovery of storage stack implementations may occasionally be obtained using vendor provided APIs (application program interfaces) or CLIs (command line interfaces), but mostly it requires performing reverse engineering. Such reverse engineering performing disk binary reads and trying to decipher how the block level processes work. This is not only time-consuming and costly, but may provide a solution that does not cover all likely usage scenarios and may not work for future releases.